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Abstract 

We have calculated the Tsallis entropy and Fisher information matrix (entropy) 
of spatially-correlated nonextensive systems, by using an analytic non-Gaussian 
distribution obtained by the maximum entropy method. Effects of the correlated 
variability on the Fisher information matrix are shown to be different from those on 
the Tsallis entropy. The Fisher information is increased (decreased) by a positive 
(negative) correlation, whereas the Tsallis entropy is decreased with increasing an 
absolute magnitude of the correlation independently of its sign. This fact arises from 
the difference in their characteristics. It implies from the Cramer- Rao inequality 
that the accuracy of unbiased estimate of fluctuation is improved by the negative 
correlation. A critical comparison is made between the present study and previous 
ones employing the Gaussian approximation for the correlated variability due to 
multiplicative noise. 
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1 Introduction 



It is well known that the Tsallis entropy and Fisher information entropy (matrix) are 
very important quantities expressing information measures in nonextensive systems. The 
Tsallis entropy for iV-unit nonextensive system is defined by [l]-[3] 

(I - c {N) ) 



with 



c f) = hpWaxiWTkdXi, (2) 



where q is the entropic index (0 < q < 3), and p^ N '({xi}) denotes the probability dis- 
tribution of N variables {xj}. In the limit of q — > 1, the Tsallis entropy reduces to the 
Boltzman-Gibbs-Shannon entropy given by 

S[ N) = - fp W ({xi}) Inp^dx^Uidxi. (3) 



The Boltzman-Gibbs-Shannon entropy is extensive in the sense that for a system consist- 
ing N independent but equivalent subsystems, the total entropy is a sum of constituent 
subsystems: Si = NS± . In contrast, the Tsallis entropy is nonextensive: S^ ^ NSfp 
for q ^ 1.0, and | q — 1 | expresses the degree of the nonextensivity of a given system. 
The Tsallis entropy is a basis of the nonextensive statistical mechanics, which has been 
successfully applied to a wide class of systems including physics, chemistry, mathematics, 
biology, and others [3]. 

The Fisher information matrix provides us with an important measure on information 
[1]. Its inverse expresses the lower bound of decoding errors for unbiased estimator in 
the Cramer-Rao inequality. It denotes also the distance between the neighboring points 
in the Rieman space spanned by probability distributions in the information geometry. 
The Fisher information matrix expresses a local measure of positive amount of informa- 
tion whereas the Boltzman-Gibbs-Shannon-Tsallis entropy represents a global measure of 
ignorance [1]. In recent years, many authors have investigated the Fisher information in 
nonextensive systems [5]-[T7]. In a previous paper [17J, we have pointed out that two types 
of generalized and extended Fisher information matrices are necessary for nonextensive sys- 
tems [T7j. The generalized Fisher information matrix obtained from the generalized 
Kullback-Leibler divergence in conformity with the Tsallis entropy, is expressed by 



4f } = & 



90* J [ de j 

2 



(4) 



where E[- ■ •] denotes the average over P^ N '{{xi}) [= P {@k})} characterized by a 

set of parameters {8k}- On the contrary, the extended Fisher information matrix 
derived from the Cramer-Rao inequality in nonextensive systems, is expressed by [17] 



c^ N ' being given by Eq. (121) . In the limit of q = 1.0, both the generalized and extended 
Fisher information matrices reduce to the conventional Fisher information matrix. 

Studies on the information entropies have been made mainly for independent (uncor- 
rected) systems. Effects of correlated noise and inputs on the Fisher information matrix 
and Shannon's mutual information have been extensively studied in neuronal ensembles 
(for a recent review, see Ref. [18]; related references therein). It is a fundamental prob- 
lem in neuroscience to determine whether correlations in neural activity are important for 
decoding, and what is the impact of correlations on information transmission. When neu- 
rons fire independently, the Fisher information increases proportionally to the population 
size. In ensembles with the limited-range correlations, however, the Fisher information 
is shown to saturate as a function of population size [H]-[2T]. In recent years the inter- 
play between fluctuations and correlations in nonextensive systems has been investigated 
[22]-[21]- It has been demonstrated that in some globally correlated systems, the Tsal- 
lis entropy becomes extensive while the Boltzman-Gibbs-Shannon entropy is nonextensive 
[22] . Thus the correlation plays important roles in discussing the properties of information 
entropies in nonextensive systems. 

It is the purpose of the present paper to study effects of the spatially-correlated vari- 
ability on the Tsallis entropy and Fisher information in nonextensive systems. In Sec. 
2, we will discuss information entropies of correlated nonextensive systems, by using the 
probability distributions derived by the maximum entropy method (MEM). In Sec. 3, 
we discuss the marginal distribution to study the properties of probability distributions 
obtained by the MEM. Previous related studies are critically discussed also. The final 
Sec. 4 is devoted to our conclusion. In appendix A, results of the MEM for uncorrelated, 
nonextensive systems are briefly summarized [EJ [101 [T7J, [25] . 




where E q [- ■ •] expresses the average over the escort probability Pq N \{xi}) given by 



Cq 



(6) 
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2 Correlated nonextensive systems 
2.1 The case of N = 2 



We consider correlated iV-unit nonextensive systems, for which the probability distribu- 
tion is derived with the use of the MEM under the constraints given by 



1 



<j- 



s a 



i 



(7) 
(8) 

(9) 
(10) 



/i, a 2 and s expressing the mean, variance, and degree of the correlated variability, re- 
spectively. Cases with N = 2 and arbitrary iV will be separately discussed in Sees. 2.1 
and 2.2, respectively. 

For a given correlated nonextensive system with N = 2, the MEM with constraints 
given by Eqs. (l7l)- (fT0~l) yields (details being explained in appendix B) 



with 



p (2) (xi,x 2 ) 



1 



K2) 



exp. 



2 2 



Z/ i=lj=l 



'111 



.4 



-(2) 



,(2) 



aSij + b (1 - 8ij), 
1 

4V(l-s 2 )' 
s 

~i/fV(l-s 2 )' 



(g-1) " V2'g-1 

27ra 2 rJ 2) , 



2/ U'g-l 



(12) 
(13) 

(14) 

for 1 < q < 3 (15) 



v/r^ 2 , 

(2-g), 



2'1-g J U'l-g 2, 



for g = 1 

for < g < 1 



(16) 

(17) 

(18) 
(19) 



where B(x,y) denotes the beta function and exp g (x) expresses the g-exponential function 

4 



defined by 

exp q (x) = [1 + (1 - q)x\ l/{1 - q) . (20) 

The matrix A with elements is expressed by the inverse of the covariant matrix Q 
given by 



(21) 



with 



Qi 



u^a% 3 + 3(1-5^)]. forz,j = l,2 



(22) 



In the limit of q = 1.0, the distribution p' 2 '(xi,X2) reduces to 

1 



p (2) (xi,x 2 ) 



2na 2 Vl - s 2 



cxp 



1\ 2 2 

z/ <=ij=i 



(23) 



which is nothing but the Gaussian distribution for N = 2. 

We have calculated information entropies, by using the distribution given by Eq. ( fTTl) . 

Tsallis entropy 

We obtain 



9(2) 



[1 + log(27T(T 5 

1 - cl 2 ) 



log(rf); 



q-l 



for g = 1 
for q 7^ 1 



(24) 
(25) 



with 



.(2) 



(2)^(2)^l- 9 



q \"q 



(26) 



where Z^ 2 ) is given by Eq. (|15|) - (|17p . From r q 2 ^ given by Eq. ffTHj) . we may obtain the s 
dependence of as given by 



c q 2 \s) 



1 + ^ 2 



- C J»>(0) 



which yields 



sS 2) (o) - 



4 2) (o) 2 



s . 



for I s |<C 1 



(27) 
(28) 



for I s I'C 1 



(29) 



5 



Figure [T]( a) shows S^/N as a function of the correlation s for N = 2 (values of 
fi = 0.0 and a 2 = 1.0 are hereafter adopted in model calculations shown in Figs. [HIT]). We 
note that Tsallis entropy is decreased with increasing absolute value of s independently 
of its sign. 

Fisher information 

By using Eqs. fll]) and ([5]) for 6i = 8j = fi, we obtain the Fisher information matrices 
given by 

9? = 2 V ( 3 °) 
H cH 1 + s) 



^ ( 2g -l)(2-g)a2(l + s )' 



which show that is independent of q and that the inverses of both matrices are 
proportional to cr 2 (l + s). 

Figure [2] shows the s dependence of the extended Fisher information for N = 2, whose 
inverse is increased (decreased) for a positive (negative) s, depending on a sign of s in 
contrast to S q . 

2.2 The case of arbitrary N 

It is possible to extend our approach to the case of arbitrary N, for which the MEM with 
the constraints given by Eqs. fl7|)- f|T0|) lead to the distribution given by (details being 
given in appendix B) 



z, 



(TV) 
Q 



exp. 



N N 



Z/ i=ii=i 



(32) 



with 



Aij = a 6ij + 6(1 - 6ij), (33) 
ll + (N-2)s] (34) 

(35) 



s 

~ 4 N) a 2 (l-s)[l + {N -l)s] 



(2u^a 2 ) N / 2 rW /l 1 A 

^ ^ ^^G'^T-s)' fOTl< ^ <3 (36) 

= (2vra 2 ) iV / 2 rf), for q = 1 (37) 
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v. 



(l-q)N/2 -*=>■- \2 7 l-q 

f) = {(l-sf^[l + (N-l)s}}^ 
{N) [(N + 2) - jVg] 
2 



+ 



for < q < 1 (38) 
(39) 
(40) 



The matrix A is expressed by the inverse of the covariant matrix Q whose elements are 
given by 



Qij = vj l N) a 2 [6 ij + 8(1-6^)). 



(41) 



In the limit of q = 1.0, the distribution given by Eq. ( 1321 becomes the multivariate 
Gaussian distribution given by 



p{{xi}) oc exp 



(42) 



It is necessary to note that there is the condition for a physically conceivable s value 
given by [see Eq. (IC8I) . details being discussed in appendix C] 



sl < s < Su, 



(43) 



where the lower and upper critical s values are given by sl = —1/(N — 1) and su = 1.0, 
respectively. In the case of iV = 2 and iV = 10, for example, we obtain sl = —1.0 and 
sl = —0.11, respectively. 

By using the probability distribution given by Eq. (|32|) . we have calculated information 
entropies whose s dependences are given as follows. 



Tsallis entropy 

We obtain 



with 



q(n) 



N 



[l + log(27nx 2 )]+log(r 



q )l 



1 - C W 

q-1 ' 



AN) 



- 4 N \o) 



x + {q-\)N{N-\) s2 



for q ■ 
for q 7^ 1 



for | s |<2/ JN(N -1 



(44) 
(45) 

(46) 
(47) 
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where the s dependence of cj^ arises from a factor of r^ N ' in Eq. dSSJ), and cf\0) 
expresses the s = 0.0 value of c^ N \ Equation (T46l) yields the s dependent given by 



Sf>(«) * 5f)(0) 



N(N-l)cf)(oy 



for | s |< 2/yjN(N- 1) (48) 



4 

where S^(0) stands for the Tsallis entropy for s = 0.0. The region where Eqs. (T47j) and 
( 148]) hold becomes narrower for larger N. 

The s dependence of S^/N for N = 10 is shown in Fig. QJb), where S^/N has a 
peak at s = 0.0 and it is decreased with increasing | s |. Comparing Fig. Q](b) with Fig. 
QJa), we notice that s-dependence of S^/N for iV = 10 is more significant than that for 
N = 2 [Eq. flSD]. 

Circles in Figs. E^a) and E£b) show with s = 0.0 for q < 1.0 and g > 1.0, 

respectively, which are calculated with the use of the expressions given by Eqs. (145!) and 
( f4T)|) . They are in good agreement with dashed curves showing exact results which are 
given by Eq. (1A13|) and shown in Figs. [3(a) and[7](b) in appendix A. Squares show S^/N 



with s = 0.5 calculated by using Eqs. (14"5]) and (146]) . The Tsallis entropy is decreased by 
an introduced correlation. Because of a computational difficulty [26j . calculations using 
Eqs. ( 1451) and (145]) cannot be performed for larger A" than those shown in Figs. 0(a) and 
Mb). 

Fisher information 

The generalized and extended Fisher information matrices are given by 

„W = - (49) 

9q a 2 [l + (N-l)sY K } 



~JN) = Nq(q + 1) 

Uq ( r 2 (3-g)(2g-l)[l + (iV-l) s ]- 



The results for q = 1.0 given by Eqs. (149 j) and (I50p are consistent with those derived with 
the use of the multivariate Gaussian distribution [19J. By using the Fisher information 
matrices for N — 1, gfp and g^\ given by Eqs. (1A17|) and (IA18j) . we obtain 



4 1} _ ~9? 



9^ ~d N) 



1 

iV + 




8, 


s, 


for N - 


-> oo 


1 

N' 


for s - 


= 0, 


1, 


for s = 




0. 


for s = 





(51) 

(52) 
(53) 
(54) 
(55) 
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which holds independently of q. Inverses of the Fisher information matrices approach the 
value of s for iV — > oo, and are proportional to 1/N for s = 0.0. In particular, they 
vanish at s = s^. These features are clearly seen in Fig. H] where the inverses of Fisher 
information matrices, gf'/gi N ^ and g^' /gi N \ are plotted as a function of N for various s 
values. 



3 Discussion 



3.1 Marginal distributions 

In the present study, we have obtained the probability distributions, applying the MEM to 
spatially-correlated nonextensive systems. We will examine our probability distributions 
in more detail. The X\ dependences of p( 2 >(xi, x 2 ) f° r N = 2 given by Eq. (TIT!) with 
s = 0.0 and s = 0.5 are plotted in Figs. 0(a) and[5^b), respectively, where x 2 is treated 
as a parameter. When s = 0.0, the distribution is symmetric with respect of x% for all 
x 2 values. When the correlated variability of s — 0.5 is introduced, peak positions of the 
distribution appear at finite x\ for X2 = 0.5 and 1.0. 

In the limit of s = 0.0 (i.e. no correlated variability), p^ 2 \xi,x 2 ) given by Eq. ( TTTT) 
becomes 



p {2) (x l , x 2 ) oc 



1 - 



[l- q )(xi + x^ 1/{1 - q) 



(56) 



which does not agree with the exact result (except for q = 1.0), as given by 



p^(x l )p^(x 2 ) 



oc 



1 - 



^ p (2) (x 1 ,x 2 ) 



(l-q)(x\ + xl) , (l-q) 



1 



+ 



4(4 1} ) 2 ^ 4 



2 2 

X X 2 



(57) 
(58) 



because of the properties of the g-exponential function defined by Eq. (TSUI) : exp q (x + y) 7^ 
exp q (x) exp q (y). By using the (/-product ® g defined by [27] 



x ® q y = [x 1 q + y 1 q 



we may obtain the expression given by 

P (1) (^i) ® q p {1) (x 2 ) 



oc 



'l-q)(x\ + xl) 



i/(i-y) 



(59) 



(60) 



which coincides with z/ 2 ) 



given by Eq. ( J56l) besides a difference between z/^ 1 ^ and 



(2) 



In order to study the properties of the probability distribution of p( 2 \xi, x 2 ) in more 
details, we have calculated its marginal probability (with s = 0.0) given by 



p( 2 \xi, x 2 ) dx 2 oc 



2i/,«V 



l/(l- ff )+l/2 



(61) 



Dashed curves in Figs. [6](a) and Mjo) show pff(xi) in linear and logarithmic scales, 
respectively. The marginal distributions are in good agreement with solid curves showing 
pWfci) [Eq. flM])], 

1/(1-9) 



p^\xi) OC 



1 _ (i - gK 



2l/JV 



(62) 



In the case of N = 3, the distribution given by Eq. (1321) yields its marginal distribution 
(with s = 0.0) given by 



p^(x 1 ,x 2 ,x 3 ) dx 2 dx 3 oc 



[} ~ g)gj 
2i/fV 



i/(i-q)+i 



(63) 



Chain curves in Figs. EJ^a) and^b) represent pffi(xi), which is again in good agreement 
with solid curves showing p^(xi). These results justify, to some extent, the probability 
distribution adopted in our calculation. 

The marginal distribution for an arbitrary iV (with s = 0.0) is given by 



oc 



■oc 



p( N \xi, xn) dx 2 ■ ■ dxN 

(l-g)xf 1 1/(1 - g)+(JV - 1)/2 
2v^a 2 



1 - 



1 - g N )x\ 



V(1-9at) 



(64) 
(65) 

(66) 



with 



qN 
v N 



(N- 1) - (N-3)q 
(jV + l)-(JV-l)g' 
(N + 2) — Nq 



(67) 
(68) 



(JV + l)-(JV-l)g' 

Equations (I66 j) -( l68j) show that in the limit of iV — > oo, we obtain q^ = 1.0, z/jv = 1.0, and 
Pm\ x i) reduces to the Gaussian distribution. 
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3.2 Comparison with related studies 

One of typical microscopic nonextensive systems is the Langevin model subjected to 
multiplicative noise, as given by [28]-[30] 
dx ■ 

= -\ Xl + p£.(t) +axi Vi (t) + H{I). (i = l-N) (69) 

Here A expresses the relaxation rate, H(I) denotes a function of an external input /, 
and a and j3 stand for magnitudes of multiplicative and additive noise, respectively, with 
zero-mean white noise given by rji(t) and £j(t) with the correlated variability, 

(vMvjit')) = a 2 [5 t3 + c M (l-5 l3 )}5(t-t'), (70) 
&(0> = P 2 [5 l3 +c A {l-5 l3 )]5{t-t'), (71) 
(tfc(f)&(0> = 0, (72) 

where ca and cm express the degrees of correlated variabilities of additive and multiplica- 
tive noise, respectively. The Fokker-Planck equation (FPE) for the probability distribu- 
tion p({xfc}, t) (= p) is given by 

+ YEE^+^^-y^^fep). (73) 

in the Stratonovich representation. 

For additive noise only (a = 0), the stationary distribution is given by 

p{{xi}) oc exp[-^^(xi - Hi){Q~ l )ij{xj - fij)], (74) 

where /ij denotes the average of Xi , and Q expresses the covariance matrix given by 

Q l3 = f3% j + c A (l-8 ij )}. (75) 

When multiplicative noise exists (a ^ 0.0), the calculation of even stationary dis- 
tribution becomes difficult, and it is generally not given by the Gaussian. Indeed, the 
stationary distribution for non-correlated multiplicative noise with a ^ 0.0, (3 ^ 0.0 and 
c A = c M = 0.0 is given by pi] [28]- [30] 



p{{xi}) oc Hi 



;i-<?) 
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V(i-g) 



e Y{x '\ (76) 



with 



1 + 



2« 2 



2A + a 2 



Ylxi 



2\ + a 2 
(2H S 



aft 



tan 



(77) 
(78) 
(79) 



P J 

The probability distribution given by Eq. (1751) for -ff = (ca = cm = 0) agrees with that 
derived by the MEM for 2 = v^cr 2 [Eq. (TA~5]) ]. For a ^ 0.0, /3 = 0.0 and if = /i > 
(ca = cm = 0), Eq. ( 1761) becomes [T7| 



p(x) 



oc 



yielding the Fisher information given by 



-2/(q-l) e ~2fi/a 2 x 



Nq A 2NXq 4 



a 2 fj? 



10) 



(81) 



where a 2 = a 2 ji 2 /2\ and 0(x) is the Heaviside function. 

The probability distribution for correlated multiplicative noise (a ^ 0.0, cm ^ 0.0) 
is also the non-Gaussian, which is easily confirmed by direct simulations of the Langevin 
model with N = 2 [31]. In some previous studies pj5]-[2T], the stationary distribution of 
the Langevin model subjected to correlated multiplicative noise with cm ^ 0.0, (3 = 0.0 
and H = fi is assumed to be expressed by the Gaussian distribution with the covariance 
matrix given by 



Qi 



This is equivalent to assume that 



<? M \ii Hj[Sij + c M (l - 5ij)]. 



(82) 



_d_ 



.1 



_d_ 



(Xj p) 



_d_ 

dx-i 



d 



d Xj 



d 2 p 



1 dxidxj ' ^ 

in the FPE given by Eq. (1751) with ^ = (xi) and a 2 M = a 2 . By using such an approxima- 
tion, Abbott and Dayan (AD) [19] calculated the Fisher information matrix of a neuronal 
ensemble with the correlated variability, which is given by 



(AO 
9 AD 



NK 



a 2 M [l + (N-l)c M ] 
N 



+ 2NK, 
2N 



a 2 Mf i 2 [l + (N-l)c M ] 
12 



+ 



14) 



with a spurious second term (2NK), where K = iV -1 J]j [d\nH(fii)/dfii] 2 = l//i 2 [Eq. 
(4.7) of Ref. [19] in our notation]. Equation (1B4"|) is not in agreement with either Eq. 
(149]) or (|50|) derived by the MEM. Furthermore, the result of AD in the limit of cm = 0, 
9 ad = (^V !// 12 + 2A^//i 2 ), does not agree with the exact result given by Eq. (1STT) 
for the Langevin model. This fact casts some doubt on the results of Refs. [T9"]-[2T] 
based on the Gaussian approximation given by Eq. fl82l) or fl83l) . which has no physical or 
mathematical justification. The Fisher information matrix depends on a detailed structure 
of the probability distribution because it is expressed by the derivative of the distribution 
with respect to x, as given by 

- d 2 lnp (A0( x 



dx 2 



5) 



We must take into account the non-Gaussian structure of the probability distribution in 
discussing the Fisher information of nonextensive systems. 



4 Conclusion 

We have discussed effects of the spatially-correlated variability on the Tsallis entropy and 
Fisher information matrix in nonextensive systems, by using the probability distribution 
derived by the MEM. Although the obtained analytical distribution in the limit of s = 0.0 
does not hold the relation given by p^ N \{xk}) = p^(xi) for q ^ 1.0, it numerically 
yields good results (Fig. [3]), reducing to the multivariate Gaussian distribution in the 
limit of q = 1.0. Our calculations have shown that 

(i) the Tsallis entropy is decreased by both positive and negative correlations, and 

(ii) the inverses of the Fisher information matrices are increased (decrease) by a positive 
(negative) correlation. 

The difference between the s dependences of S q and g q arises from the difference in their 
characteristics: S q provides us with a global measure of ignorance while g q a local measure 
of positive amount of information [4]. The item (ii) implies that the accuracy of unbiased 
estimate of fluctuation is improved by the negative correlation. If there is known, and 
strong, negative correlation between successive pairs of data, estimating the unknown 
parameter as their simple average must reduce the error, as negatively correlated errors 
tend to cancel in taking the difference. 

In connection with the discussion presented in Sec. 3, it is interesting to make a 
detailed study on the properties of information entropies in the Langevin model subjected 
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to correlated as well as uncorrelated multiplicative noise. Such a calculation is in progress 
and will be reported elsewhere [31]. 
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Appendix A: MEM for non-correlated nonextensive systems 

We summarize results of the MEM for nonextensive systems [HI El QUI El E5] • In order 
to apply the MEM to iV-unit nonextensive systems, we impose the three constraints given 
by 



a 



JpWdxiVUidXi.. 
E E i N ' 

i 

Y,E q \(Xi - fif 



1 

N 

1 

N 



(Al) 
(A2) 

(A3) 



where the g-dependent /x and a 2 correspond to p q and a 2 , respectively, in [T7j . For a given 
nonextensive system, the variational condition for the Tsallis entropy given by Eq. (0Q) 
with the constraints flAlj) - flA3j) yields the g-Gaussian distribution given by [61 191 [TUl \TT\ 125] 



with 



P {l \x) 



V 



(TV) 



^yexp 9 



({ Xk }) = n£, lP w. 



(x- jlf 



X ■/. 



exp„ 



V 2i/JV 



(1) I dx, 



'H 1] * 2 ) l/2 E (i J__l' 

27TCT, 



V 1-9 

3-9 



2 ! 1-q 



for 1 < g < 3 
for g = 1.0 
for < q < 1 



(A4) 

(A5) 
(A6) 

(A7) 
(A8) 
(A9) 

(A10) 
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where B(x,y) denotes the beta function. 



Tsallis entropy 

Substituting the probability distribution given by Eq. ( IA4I) to Eqs. (CQ) and (j2j), we 
obtain the Tsallis entropy expressed by 

SW = y[l + log(27ra 2 )], for q = l (All) 

= [ . [ \ K forg^l (A12) 

where c« = i/W^ 1 )) 1 " 9 . We may express Sf^ in terms of Sjp by [T7j 

TV 



= T,^(-l) k -\q-l) k '\S^) k , (A13) 

k=l 

= NSU- N{N -l ){q - 1 \ sMf + ..., (A14) 



2 



where = N\/(N — A;)! k\. Equation (1A14|) clearly shows that the Tsallis entropy is 
generally nonextensive except for q = 1.0 for which = NSjp. Figures [TJ^a) and 

W[h) show the N dependence of the Tsallis entropy per element, SfW /N, of uncorrelated 
systems (s = 0.0), which are calculated with the use of Eq. flA13j) . With increasing N, 



S^/N is decreased for q > 1.0 whereas it is significantly increased for q < 1.0. 

The Fisher information 

With the use of Eqs. fll]) and (JSJ) for 0, = 9j = the generalized and extended Fisher 
information matrices are given by [17] 

9 { q N) = NgU, (A15) 

gM = NgV (A16) 

with 

9? = 4 (A17) 



& = ( 3 -;" ( 2, -o," (Ai8) 

which show that Fisher information matrices are extensive. 
Appendix B: MEM for correlated nonextensive systes 
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In the case of N = 2, the probability distribution p^(xi,x 2 ) given by Eqs. (fTTi) and 
( [12]) is rewritten as 



p {2) (x u x 2 ) oc [l-(l-q)^ 2 \x 1 ,x 2 )} 1 ^\ (Bl) 



with 



® (2) (x 1 ,x 2 ) = - [a(xl + x\) + 2bx 1 x 2 ] , (B2) 

= Wi + ^yl (B3) 

where Aj and ?/j (z = 1,2) are eigenvalues and eigenvectors of $(xi,x 2 ) given by 



Ai = 


2(0 + 6), 


(B4) 


A 2 = 




(B5) 


Vx = 




(B6) 


2/2 = 




(B7) 



The averages of -Ejy 2 ] and are given by 

^[y 2 2 ] 

from which we obtain a 2 and s a 2 as 



(2) \ 
Vq Al 



0" 



i^ + x^i^ + y 2 ], 
a 



(B8) 
(B9) 



vf\a 2 -b 2 ) 



s a 



E q [x x x 2 ] = ^E q [y 1 -y 2 ), 



u {2 \a 2 -b 2 )' 

By using Eqs. (IBIOI) and (IB11I) . a and b are expressed in terms of a 2 and s as 

1 



(BIO) 



(Bll) 



4 2 V 2 (i-s 2 )' 

s 

4 2) ^ 2 (i -s 2 ) 
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(B12) 
(B13) 



which yield the matrix of A given by an inverse of the covariance matrix of Q [Eq. ([22]) ]. 

A calculation for the case of an arbitrary A" may be similarly performed as follows. 
The distribution given by Eqs. (|32|) and (1331) is rewritten as 



with 



$ (7V) ({x 4 }) 



i i<j 



(B14) 

(B15) 
(B16) 



where A« and %ji are eigenvalues and eigenvectors, respectively. With the use of eigenvalues 
given by 

1 



A, 



Eqs. © and W lead to 



2 [a + (N -1)6] 

o( a_6 )> 



for i — 1 
for 1 < z < JV 



(B17) 
(B18) 



[a + 6(A^ - 2)] 



s a 



Vq N \a — b)[a + (N — 1)6] 



(B19) 



z/fV -6)[o + (JV- 1)6] 



(B20) 



From Eqs. (IB19[) and f]B20[) . a and 6 are expressed in terms of a 2 and s, as given by 

„ [l + (iV-2>] 



i/?V(l-s)[l + (iV-l)s]' 
s 

~ v { q N) a 2 (l-s)[l + (N - l)s] 



(B21) 
(B22) 



Appendix C: The condition for a conceivable s value 
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In order to discuss the condition for a physically conceivable s value, we consider the 
global variable X(t) defined by 

*(*) = 4 (ci) 

% 

The first and second q moments of X(t) are given by 

= ^E^[**(*)l = /*(*), ( C2 ) 

i 

E q [{5X(t)} 2 } = ijj;^^)], (C3) 

= T^EW^W} 2 ] + ^EE ^[^(t)^.(t)], (C4) 

= + -!>(*)], (C5) 

where 5xj = ^(t) — fi(t) and 5X(t) = X(t) — Since global fluctuation in X is smaller 
than the average of local fluctuation in {x^}, we obtain 

< E q [{8X(tm < ^ F] = a{tf. (C6) 

i 

Equations (JU5} and O y ield 



< ' 1 + <^ 1 > 8 ">1 < 1.0, (C7) 

which leads to 

s L < s{t) < Su , (C8) 
with s L = -1/(JV - 1) and = 1.0. 
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Figure 1: The s dependence of the Tsallis entropy per element, S^/N, with (a) N = 2 
and (b) N = 10 for various q values with [i = 0.0 and a 2 = 1.0: the s value is allowed to 
be -1.0 < s < 1.0 for N = 2, and -0.11 < s < 1.0 for iV = 10 [Eq. (j35j)]. 



Figure 2: The s dependence of the inverse of the extended Fisher information for 
various q values with N = 2 (fj, = 0.0, a 2 = 1.0). 



Figure 3: (Color online) (a) The N dependence of the Tsallis entropy per element, 
SW/N, for q < 1.0: (q,s) = (0.8,0.0) (filled circles), (0.8,0.5) (filled squares), (0.9,0.0) 
(open circles) and (0.9,0.5) (open squares), (b) S^/N for q > 1.0: (q,s) = (1.01,0.0) 
(filled circles), (1.01,0.5) (filled squares), (1.05,0.0) (open circles) and (1.05,0.5) (open 
squares). Dashed curves denote exact results given by Eq. (1A13I) [Figs. [7(a) and[7J(b)] in 
appendix A. Note the logarithmic and linear vertical scales in (a) and (b), respectively. 



Figure 4: The A" dependences of inverses of the Fisher information matrices, gf*/gi N ' 
and g^ /cjq N \ for various s values given by Eq. (JSTj) : results for s = —0.05 and s = —0.1 
are valid for A^ < 21 and N < 11, respectively. 



Figure 5: The probability distribution of A^ = 2 systems, p^ 2 \xi,X2) [Eq. (iTTj) ]. for (a) 
s = 0.0 and (b) s = 0.5 with q = 1.5 as a function of x\ for X2 = 0.0, 0.5, and 1.0. 



Figure 6: (Color online) The uncorrelated distribution for N = 1, p^(xi) [Eq. fl62l) : 
solid curves], and marginal distributions of p$(xi) for A^ = 2 [Eq. fIBTl) : dashed curves], 
and pS 5 (xi) for N = 3 [Eq. (JH3]): chain curves] with q = 0.5 and q = 1.5 in (a) linear and 
(b) logarithmic vertical scales. 



Figure 7: The A^ dependence of the Tsallis entropy per element, /N, in uncorrelated 
systems for (a) q < 1.0 and (b) q > 1.0: note the logarithmic and linear vertical scales in 
(a) and (b), respectively. 
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